1. Related Application Data
The present disclosure is related to the following copending, commonly assigned U.S. patent applications:
U.S. patent application Ser. No. 08/621,189, entitled "Machine Vision Methods for Inspection of Leads on Semiconductor Die Packages," filed in the name of Sanjay Nichani on Mar. 21, 1996 (Attorney Docket No. 150448-031);
U.S. patent application Ser. No. 08/621,190, entitled "Machine Vision Methods for Inspection of Semiconductor Die Surfaces," filed in the name of Sanjay Nichani on Mar. 21, 1996 (Attorney Docket No. 150448-032);
U.S. patent application Ser. No. 08/621,137, entitled "Machine Vision Methods for Image Segmentation Using Multiple Images," filed in the name of Sanjay Nichani on Mar. 21, 1996 (Attorney Docket No. 150448-030); and
U.S. patent application Ser. No. 08/718,674, entitled "System or Method for Detecting Defect Within a Semi-Opaque Enclosure," filed in the name of Sanjay Nichani on even date herewith (Attorney Docket No. 221757).
Each of the above-listed copending, commonly-assigned U.S. patent applications is hereby incorporated herein by reference in its entirety.
2. Reservation of Copyright
The disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the U.S. Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
3. Field of the Invention
The present invention relates to a machine vision system or method for detecting at least one object within a semi-opaque enclosure. More particularly, the present invention relates to such a system or method which obtains or enhances an object image while eliminating or diminishing a background image which accompanies the object image.
4. Description of Background Information
Computer vision, otherwise referred to as image processing, involves the extraction of vision-related information by obtaining signals representing a scene and performing image signal processing on those signals. Applications of computer vision (image processing) techniques include character recognition, industrial inspection of manufactured items, robot guidance systems, radiology, remote sensing, and so on.
Image processing methods may typically comprise several common processing steps, e.g., as described by Rafael C. Gonzalez and Paul White in a book entitled "Digital Image Processing," Addison-Wesley (1992), the content of which is hereby incorporated herein by reference in its entirety.
In a first step, image acquisition is performed to acquire an image of the scene to be analyzed. The image may, for example, be represented in the form of a monochrome or simple digital image f(x,y) discretized both in spatial coordinates x,y and in brightness (gray levels) f.
Pre-processing and enhancement techniques may then be performed on the digital image in order to improve the image and increase the chances for success in subsequent processes. Such pre-processing and image enhancement techniques may include enhancing the contrast between one or more object images and a background image, and filtering noise from the image.
Segmentation may then be performed, which involves distinguishing between different types of regions within the input image, e.g., distinguishing between an object image and a background image.
In a next step, representation and description processing may be performed. In performing representation processing, the data obtained as a result of segmentation is converted into a "representation" suitable for computer processing. In performing description processing, "feature descriptions" are extracted from the representation. Such "feature descriptions" may categorize objects to allow one class of object to be differentiated from another.
In a final stage of the image processing, recognition and interpretation processes are performed. Recognition processing may include assigning a label to an object based upon information provided by its "feature descriptions." Interpretation processing may assign a meaning to an arrangement of recognized objects. For example, several characters may be "interpreted" as forming a particular word.
Golden Template Comparison (GTC) is an image processing method that has been used to detect flaws and defects in two-dimensional scenes of an inspected item which are highly repeatable and do not suffer from geometric distortion. Such scenes are common in semiconductor production and graphic arts applications. Generally, GTC involves the comparison of a test image to an image of a known good scene referred to as a template image. More specifically, the test image is subtracted from the template image, and differences between the two images are observed in order to determine whether a flaw or defect is present within the inspected item.
Generally, GTC comprises two main phases: training and inspection. During training, the template image is constructed by sampling a plurality of images each of which represents a scene of an item absent defects. During inspection, the test image, which represents the scene of the item to be tested (the inspected item), is then compared to the template image. More specifically, the images are subtracted to form a difference image. Thresholding is performed on the difference image to produce a resulting binary image called an error image containing either background (healthy) pixels or defect pixels which represent flaws. The error image is then analyzed in order to determine whether the tested item should be labeled as an "accept" or "fail" item. The analysis that may be performed on the error image may include counting defect pixels found in the error image and performing a blob analysis on the defect pixels. The results of the analysis may be stored in a data structure called a results structure.
In various commercial settings, service providers receive from their customers items enclosed in semi-opaque envelopes, and process the enclosed items in some manner. The service provider regularly receives large numbers of such envelopes. The types of items in the envelopes vary, and the processing steps performed on the items change in accordance with the item type. Thus, the service provider separates the envelopes according to item type. To achieve this, the service provider may use an automated content identifying system to identify or classify the enclosed items.
Photograph developers receive large numbers of paper envelopes containing different types of film (e.g., "126", "110", 35 mm, and disk cartridge type films), and process (develop) each film type using a unique set of processing steps. Thus, before processing, the photograph developer separates the envelopes into groups according to the envelope's contents.
In order to identify the contents of the envelopes, a form is provided on the outside of each paper envelope which can be filled out by the customer. The form may include boxed which can be checked to categorize the contents of the envelope, for example, to indicate that the type of film in the envelope is 35 mm, 110, or a disc type film. A character recognition system or another type of image processing system may be provided to automatically ascertain the information indicated on the outside of the envelope in order to categorize the envelopes in accordance with their contents.
In other commercial settings, goods are placed in semi-opaque enclosures, for example, envelope(s) or cardboard box(es), temporarily stored and sent to a desired destination. The contents of these enclosures may be identified by placing certain characters or another type of external identifier, such as a bar code, on the outside of the enclosure. In order to quickly determine the contents of such an enclosure, a character recognition, bar code or other type of reading system may be utilized. Such systems can be problematic if an error occurs in packaging, and an enclosure contains an item different than the item identified with the enclosure's external identifier. In commercial photo-film processing settings, check-marks are put in boxes by consumers to identify the contents of the envelope. The check-mark identification may be erroneous, as the consumer may carelessly make the mark or not even provide a mark.
Accordingly, there is a significant need for a system which can quickly and automatically identify and verify the content of a semi-opaque enclosure.
5. Definitions Of Terms
The following term definitions are provided to assist in conveying an understanding of the various exemplary embodiments and features disclosed herein.
Back Lighting
Lighting produced by a light source (or emanating from a channel) directed toward an image acquisition device such as a camera, and located behind the object of interest. Back lighting causes the image acquisition device to receive a silhouette of the object of interest.
Channel
A location from which light is emanating. A channel may be a light source or it may be an optical element redirecting light from another light source. Such an optical element may include, for example, a beam splitter, or a split mirror.
Front lighting
Lighting produced by a light source (or emanating from a channel) directed toward an object of interest and generally on the same side of the object of interest as the image acquisition device.
Segmentation
A process of distinguishing between different types of regions within an image. Object/background discrimination is a particular type of segmentation which includes identifying those portions within an image corresponding to an object and those portions within the image corresponding to the background. Defect and flaw detection is a type of object/background discrimination.
Training
A process (commonly used in Golden Template Comparison flaw and defect detection methods) of obtaining a plurality (usually several) sample images of flawless items to generate both a template image and statistical information pertaining to the template image.